P a g e 1 | 16

P a g e 2 | 16

Q1. What is SuperGlue?

Answer:

SuperGlue is a Learning Feature Matching with Graph Neural Networks. Correspondences between

points in images are essential for estimating 3D structure and camera poses in geometric computer

vision(OpenCV) tasks such as SLAM(Simultaneous Localization and Mapping) and SfM( Structure-

from-Motion). Such correspondences are generally estimated by matching local features, the process

end and the bundle adjustment or poses estimation back-end, our network lies directly in middle

– SuperGlue is a learnable middle-end (see in above Figure).

In this work, learning feature matching is viewed as finding partial assignment between two sets of local

feature. We revisit classical graph-based strategy of matching by solving the linear assignment problem,

which, when relaxed to the optimal transport problem, can be solved differentiably. The cost function of

this optimization is predicted by a GNN(Graph Neural Network). Inspired by success of the Transformer,

it uses self- (intra-image) and cross- (inter-image) attention to leveraging both spatial relationships of

P a g e 3 | 16

keypoints and their visual appearance. This formulation enforces assignment structure of the prediction

while enabling cost to learn complex priors, handling occlusion, and non-repeatable keypoints. Our

method is trained end-to-end from images pair – we learn priors for pose estimation from large annotated

of indoor and outdoor pose estimation and paves the way towards end-to-end deep SLAM.

Q2. What is MixNMatch?

Answer:

P a g e 4 | 16

It is a Multifactor Disentanglement and Encoding for Conditional Image Generation. Consider the real

image of the yellow bird in the above Fig, First column. What would a bird look like in a different

background, say that of a duck? How about in the different texture, perhaps that of the rainbow

entirely new image (last column)

While research in conditional image generation has made tremendous progress, no actual work can

simultaneously disentangle background, object pose, shape, and texture with minimal supervision, so

that these factors can be combined from multiple real images for fine-grained controllable image

change,e.g., the object’s background, shape, and texture, while keeping its pose the same. Others require

intense supervision in the form of keypoint or pose or mask annotations [peng-iccv2017, Balakrishnan-

cvpr2018, ma-cvpr2018, esser-cvpr2018], which limit their scalability and still fall short of disentangling

all of four factors outlined above.

Our proposed conditional generative model, MixNMatch, aim to fill this void. MixNMatch learns to

disentangle and encode background, object pose, shape, and texture latent factors from the real images,

and importantly, does so with minimal human supervision. This allows, e.g., each factor to be extracted

from a different actual image, and then combined for mix-and-match image generation; see in above fig.

During training, MixNMatch only requires a loose bounding box around the object to the model

background but requires no other supervision for modeling the object’s pose, shape, and texture.

P a g e 5 | 16

Answer:

Figure: Visual results on four datasets. Vertically we show input in row first and our results in row third.

For LFW and SCface datasets, we show the ground truth and gallery images in second row, respectively.

For WIDER FACE and QMUL-SurFace datasets which do not have ground truth high-resolution images,

we compare with two state-of-the-art(SOTA) methods: Bulat et al. [bulatyang2018learn] and

FSRGAN [CT-FSRNet-2018] in row 2, respectively.

It is used for Surveillance Face Recognition and Normalization. Surveillance Face Recognition (FR) is

a challenge and a significant problem yet less studied. The performance on conventional benchmarks

such as LFW [LFWTech] and IJB-A have been greatly improved by state-of-the-art (SOTA) (Face

Illumination, Expression (PIE).

In this work, we study surveillance face recognition(FR) and normalization. Specifically, given the

surveillance face image, we aim to learn robust identity features for Face recognition(FR). Meanwhile,

the feature are used to generate a normalized face with enhanced facial details and neutral PIE. Our

normalization is performed mainly on the aspect of the resolution. While sharing same goal as traditional

SR, it differs in removing the pixel-to-pixel correspondence between original and super-resolved images,

as required by conventional SR. Therefore, we term it as face normalization. For same reason, we

P a g e 6 | 16

compare ours to FSR instead of prior normalization methods operating on pose or expression. To the

best of our knowledge, this is a first work to study surveillance face normalization.

We propose the novel Feature Adaptation Network(FAN) to jointly perform face recognition and

normalization, which has 3 advantages over conventional FSR. i) Our joint learning scheme can benefit

Our Feature Adaptation Network (FAN) consists of 2 stages. In first stage, we adopt disentangled features

learning to learns both identity and non-identity characteristics mainly from high-resolution(HR) images,

which are combined as input to the decoder for pixel-wise face recovering. In second stage, we propose

feature adaptation to facilitate the feature further learning from the low-resolution (LR) images by

approximating feature distribution between the low-resolution and high-resolution identity encoders.

There are two advantages to use Feature Adaption Network(FAN) for surveillance facial-

recognition(FR) and normalization. First, Feature Adaption Network (FAN) focuses on learning

disentangled identity features from Low-resolution(LR) images, which is better for facial recognition

(FR) than extracting features from super-resolved

faces [tran2017disentangled, zhang2018facesr, wu2016j]. 2nd, our adaptation is performed in

P a g e 7 | 16

correspondences. As shown in the last fig., the synthetic paired data used in prior

works [CBN_ECCV16, CT-FSRNet-

In contrast, our RSA efficiently alleviates the constraint on scale variation.

Q5. WSOD with PSNet and Box Regression

Answer:

The object detection task is to find objects belonging to specified classes and their locations in images.

Benefiting from the rapid development of deep learning(DL) in recent years, the fully supervised object

detection task has made significant progress. However, fully supervised task requires instance-level

annotation for training, which costs lot of time and resources. Unlabeled or images labeled datasets

cannot be effectively used by fully supervised method. On another hand, image-level annotated datasets

are easy to generate and can even be automatically generated by web search engines. To effectively

utilize these readily available datasets, we focus on weakly-supervised object detection(WSOD) tasks.

P a g e 8 | 16

proposal ultimately consists of the whole dog, PSNet gives a low score.

There are 3 main methods for weakly supervised object detection: The first is to update detector and

pseudo labels from inaccurate pseudo labels iteratively; The second is to construct an end-to-end network

that can take image-level annotation as supervision to train this object detection network. The third two-

stage method is that taking an algorithm to optimize pseudo labels from other WSOD networks and

training a fully supervised object detection network. In addition, according to different modes of

proposing proposals, each of above methods can be divided into 2 classes: one is to propose proposals

based on feature map that predicts probability of each pixel belonging to each class, and then get the

possible instances and their locations in image; The second is detector-based method that uses the trained

detector to identify multiple proposals and determines whether each proposal belongs to a specific object

class or not. Comparing the effects of these methods, the end-to-end detector-based approach performs

P a g e 9 | 16

significant improvement in the level of realism of generated data, increasing both resolution and quality.

Nowadays, powerful methods exist for creating an image from scratch, and for changing its style or only

artifacts will soon disappear. Other approaches rely on the fact that any acquisition device leaves

allow associating a media with its acquisition device at various levels, from the type of source (camera,

device identification is possible based on the camera photo-response non-uniformity (PRNU) pattern.

to any classifier and is also very well known in forensics, where many counter-forensics methods have

been proposed in the literature. Indeed, forensics and counter-forensics go hand in hand, a competition

that contributes to improving the level of digital integrity over time.

In this work, we propose a method to synthesize traces of cameras using a generative approach that is

agnostic to the detector (i.e., not just targeted adversarial noise). We achieve this by training a conditional

generator to jointly fool an adversarial discriminator network as well as a camera embedding network.

To this end, the proposed method injects the distinctive traces of a target camera model in synthetic

images, while reducing the first generation traces themselves, leading all tested classifiers to attribute